Overcoming Incomplete Perception with Utile Distinction Memory
نویسنده
چکیده
This paper presents a method by which a reinforcement learning agent can solve the incomplete perception problem using memory. The agent uses a hidden Markov model (HMM) to represent its internal state space and creates memory capacity by splitting states of the HMM. The key idea is a test to determine when and how a state should be split: the agent only splits a state when doing so will help the agent predict utility. Thus the agent can create only as much memory as needed to perform the task at hand|not as much as would be required to model all the perceivable world. I call the technique UDM, for Utile Distinction Memory.
منابع مشابه
Overcoming Incomplete Perception with Util Distinction Memory
This paper presents a method by which a reinforcement learning agent can solve the incomplete perception problem using memory. The agent uses a hidden Markov model (HMM) to represent its internal state space and creates memory capacity by splitting states of the HMM. The key idea is a test to determine when and how a state should be split: the agent only splits a state when doing so will help t...
متن کاملLearning Task-Relevant State Spaces with a Utile Distinction Test
This paper presents a reinforcement learning algorithm that learns an agent-internal state space on-line, in response to the demands of the task—thus avoiding the need for the agent designer to delicately engineer the agent's internal state space. The algorithm scales well with (1) large perceptual state spaces by pruning away unnecessary features, and (2) “overly small” perceptual state spaces...
متن کاملLearning to Use Selective Attention and Short-Term Memory in Sequential Tasks
This paper presents U-Tree, a reinforcement learning algorithm that uses selective attention and shortterm memory to simultaneously address the intertwined problems of large perceptual state spaces and hidden state. By combining the advantages of work in instance-based (or “memory-based”) learning and work with robust statistical tests for separating noise from task structure, the method learns...
متن کاملEfficient Exploration in Reinforcement Learning Based on Short-term Memory
Reinforcement learning addresses the question of how an autonomous agent that senses and acts in its environment can learn to choose optimal actions to achieve its goals. It is related to the problem of learning control strategies. In practice multiple situations are usually indistinguishable from immediate perceptual input. These multiple situations may require different responses from the age...
متن کاملInstance-Based Utile Distinctions for Reinforcement Learning with Hidden State
We present Utile Suffix Memory, a reinforcement learning algorithm that uses short-term memory to overcome the state aliasing that results from hidden state. By combining the advantages of previous work in instance-based (or “memorybased”) learning and previous work with statistical tests for separating noise from task structure, the method learns quickly, creates only as much memory as needed ...
متن کامل